Analyzing Online Sentiment to Predict Telephone Poll Results

The telephone survey is a common social science research method for capturing public opinion, for example, an individual's values or attitudes, or the government's approval rating. However, reducing domestic landline usage, increasing nonresponse rate, and suffering from response bias of the interviewee's self-reported data pose methodological challenges to such an approach. Because of the labor cost of administration, a phone survey is often conducted on a biweekly or monthly basis, and therefore a daily reflection of public opinion is usually not available. Recently, online sentiment analysis of user-generated content has been deployed to predict public opinion and human behavior. However, its overall effectiveness remains uncertain. This study seeks to examine the temporal association between online sentiment reflected in social media content and phone survey poll results in Hong Kong. Specifically, it aims to find the extent to which online sentiment can predict phone survey results. Using autoregressive integrated moving average time-series analysis, this study suggested that online sentiment scores can lead phone survey results by about 8-15 days, and their correlation coefficients were about 0.16. The finding is significant to the study of social media in social science research, because it supports the conclusion that daily sentiment observed in social media content can serve as a leading predictor for phone survey results, keeping as much as 2 weeks ahead of the monthly announcement of opinion polls. We also discuss the practical and theoretical implications of this study.